[SPARK-56912][SQL] Simplify Cast to boolean codegen under ANSI mode by gengliangwang · Pull Request #55937 · apache/spark

gengliangwang · 2026-05-17T23:30:21Z

What changes were proposed in this pull request?

Extend UTF8StringUtils.scala with toBooleanExact(UTF8String, QueryContext) and use it from Cast.scala for the ANSI String -> Boolean cast path (both eval and codegen), alongside the peer string ANSI helpers (toByteExact / toShortExact / toIntExact / toLongExact). The non-ANSI path keeps the inline if/else if/else evNull = true form because it has no error to throw.

Why are the changes needed?

Part of SPARK-56908 (umbrella). The ANSI String->Boolean cast emits an 8-line if (isTrueString) … else if (isFalseString) … else throw block in codegen. This PR collapses it to a one-line UTF8StringUtils.toBooleanExact(...) call.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

build/sbt "catalyst/testOnly *CastSuite *CastWithAnsiOnSuite *CastWithAnsiOffSuite *AnsiCastSuite *TryCastSuite"

307/307 pass.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Cursor 1.x

gengliangwang · 2026-05-17T23:32:31Z

Stack overview (SPARK-56908 umbrella)

This PR is part of the SPARK-56908 codegen-simplification series. Current status:

Merged:

[SPARK-56909][SQL] Simplify Cast to int/long codegen under ANSI mode #55934 — [SPARK-56909][SQL] Simplify Cast to int/long codegen under ANSI mode
[SPARK-56913][SQL] Simplify BinaryArithmetic byte/short codegen under ANSI mode #55938 — [SPARK-56913][SQL] Simplify BinaryArithmetic byte/short codegen under ANSI mode
[SPARK-56915][SQL] Simplify MakeDate/MakeInterval codegen under ANSI mode #55940 — [SPARK-56915][SQL] Simplify MakeDate/MakeInterval codegen under ANSI mode
[SPARK-56916][SQL] Simplify ElementAt array codegen under ANSI mode #55941 — [SPARK-56916][SQL] Simplify ElementAt array codegen under ANSI mode
[SPARK-56935][SQL] Simplify GetArrayItem codegen and consolidate ElementAtUtils into ArrayExpressionUtils #55973 — [SPARK-56935][SQL] Simplify GetArrayItem codegen and consolidate ElementAtUtils into ArrayExpressionUtils

Open:

[SPARK-56910][SQL] Simplify Cast to byte/short codegen under ANSI mode #55935 — [SPARK-56910][SQL] Simplify Cast to byte/short codegen under ANSI mode (branches off master)
[SPARK-56911][SQL] Simplify Cast to decimal codegen under ANSI mode #55936 — [SPARK-56911][SQL] Simplify Cast to decimal codegen under ANSI mode (stacked on [SPARK-56910][SQL] Simplify Cast to byte/short codegen under ANSI mode #55935)
[SPARK-56912][SQL] Simplify Cast to boolean codegen under ANSI mode #55937 — [SPARK-56912][SQL] Simplify Cast to boolean codegen under ANSI mode (stacked on [SPARK-56911][SQL] Simplify Cast to decimal codegen under ANSI mode #55936)
[SPARK-56914][SQL] Simplify decimal arithmetic codegen under ANSI mode #55939 — [SPARK-56914][SQL] Simplify decimal arithmetic codegen under ANSI mode (stacked on [SPARK-56911][SQL] Simplify Cast to decimal codegen under ANSI mode #55936)
[SPARK-57004][SQL] Simplify CheckOverflowInSum codegen under ANSI mode #56063 — [SPARK-57004][SQL] Simplify CheckOverflowInSum codegen under ANSI mode (independent)

gengliangwang · 2026-05-21T18:32:33Z

Audited this PR for the same lessons surfaced by @viirya and @cloud-fan on #55938 (and applied to #55934 / #55939):

Are the helpers redundant with an existing Scala API? No — there's no existing Scala object that wraps StringUtils.isTrueString / isFalseString with the ANSI invalidInputSyntaxForBooleanError throw. The helper is net-new.
Are the eval-path additions redundant? No — master ANSI string -> boolean was a 7-line inline block. The new ANSI branch is a one-line helper call; the non-ANSI branch is simplified by moving the ansiEnabled check out into the case clause.

So no changes needed on this PR for that review.

### What changes were proposed in this pull request? Extend `CastUtils.java` with `stringToBooleanExact(UTF8String, QueryContext)` and use it from `Cast.scala` for the ANSI `String -> Boolean` cast path (both eval and codegen). The non-ANSI path keeps the inline `if/else if/else evNull = true` form because it has no error to throw. ### Why are the changes needed? Part of SPARK-56908 (umbrella). The ANSI String->Boolean cast emits an 8-line `if (isTrueString) … else if (isFalseString) … else throw` block in codegen. This PR collapses it to a one-line `CastUtils .stringToBooleanExact(...)` call. ### Does this PR introduce _any_ user-facing change? No. The compiled behavior is identical; only the emitted Java source text changes. ### How was this patch tested? ``` build/sbt "catalyst/testOnly *CastSuite *CastWithAnsiOnSuite \ *AnsiCastSuite *TryCastSuite" ``` 204/204 pass. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor 1.x

cloud-fan

This PR is part of the SPARK-56908 codegen-simplification stack (described in your first conversation comment). The ANSI String→Boolean cast previously emitted an 8-line if/else if/else throw block in codegen and a parallel block in the eval path. The PR collapses both into a single helper call.

Prior state. castToBoolean (eval) and castToBooleanCode (codegen) each had a single _: StringType case that branched on ansiEnabled to choose between throwing invalidInputSyntaxForBooleanError and assigning null / evNull = true. The ANSI throw path duplicated the helper-method machinery from peer ANSI casts.

Design approach. Add a stringToBooleanExact(UTF8String, QueryContext) helper that performs the isTrueString → isFalseString → throw sequence and call it from a new case _: StringType if ansiEnabled branch in both castToBoolean and castToBooleanCode. The non-ANSI branch keeps the inline if/else if/else because there's no error to throw — same separation that peer casts (castToByteCode, castToIntCode, etc.) already use.

Key design decision — where the helper lives. The PR places it in CastUtils.java next to the integral/fractional narrowing helpers, but the peer string→primitive ANSI helpers (toByteExact / toShortExact / toIntExact / toLongExact) live in UTF8StringUtils.scala. See the inline comment recommending the move — it preserves the string-source/numeric-source split that UTF8StringUtils and CastUtils already maintain, and keeps CastUtils's class Javadoc accurate.

Implementation sketch. Behavior is preserved across both eval and codegen paths: the helper executes the same isTrueString / isFalseString / throw sequence; the non-ANSI inline branch keeps the same isTrueString / isFalseString / null sequence. Existing test coverage (CastSuite / CastWithAnsiOnSuite / CastWithAnsiOffSuite / TryCastSuite, plus the postgreSQL/boolean.sql.out golden file) exercises both paths.

cloud-fan · 2026-05-28T11:50:57Z

    return d.changePrecision(precision, scale) ? d : null;
  }
+
+  // ----- string -> boolean (ANSI: throw on invalid syntax) -----


Move stringToBooleanExact to UTF8StringUtils.scala — the peer string→primitive ANSI helpers (toByteExact / toShortExact / toIntExact / toLongExact) already live there. Renaming to toBooleanExact(UTF8String, QueryContext): Boolean for symmetry, and switching the codegen call site to UTF8StringUtils.getClass.getCanonicalName.stripSuffix("$") like the peers (e.g. castToByteCode at Cast.scala:2063), gives a fully consistent shape.

This also keeps the CastUtils.java class Javadoc accurate — its current "for ANSI overflow-checked casts" line and the per-row references[] rationale apply to the integral/fractional narrowing helpers but not to a string→boolean invalid-input check.

Done in 83a3cbb112a:

Added UTF8StringUtils.toBooleanExact(UTF8String, QueryContext): Boolean next to the existing toByteExact / toShortExact / toIntExact / toLongExact peers.

Switched the eval and codegen call sites in Cast.scala (castToBoolean / castToBooleanCode) to use it via the getCanonicalName.stripSuffix("$") pattern, matching castToByteCode etc.

Removed stringToBooleanExact from CastUtils.java along with its UTF8String / StringUtils imports, so the class Javadoc's "ANSI overflow-checked casts" + per-row references[] framing stays accurate.

Updated the UTF8StringUtils class doc from "casting string to numeric values" to "casting string to primitive values under ANSI mode" so the new boolean helper is in scope.

lint-scala clean, 307/307 Cast tests pass.

…s.toBooleanExact, peer the other string ANSI helpers

dongjoon-hyun

Is the PR description correct, @gengliangwang ? stringToBooleanExact in this PR?

Extend CastUtils.java with stringToBooleanExact(UTF8String, QueryContext) and use it from Cast.scala for the ANSI String -> Boolean cast path (both eval and codegen). The non-ANSI path keeps the inline if/else if/else evNull = true form because it has no error to throw.

gengliangwang · 2026-05-29T21:39:05Z

Good catch, @dongjoon-hyun — the description was stale. The helper was moved/renamed in the follow-up commit (stringToBooleanExact in CastUtils.java → UTF8StringUtils.toBooleanExact, peering the other string ANSI helpers). I've updated the PR description to match the current code.

dongjoon-hyun

+1, LGTM. Thank you, @gengliangwang and @cloud-fan .

gengliangwang · 2026-05-29T22:16:48Z

@cloud-fan @dongjoon-hyun Thanks for the review. Merging this one to master/4.x

### What changes were proposed in this pull request? Extend `UTF8StringUtils.scala` with `toBooleanExact(UTF8String, QueryContext)` and use it from `Cast.scala` for the ANSI `String -> Boolean` cast path (both eval and codegen), alongside the peer string ANSI helpers (`toByteExact` / `toShortExact` / `toIntExact` / `toLongExact`). The non-ANSI path keeps the inline `if/else if/else evNull = true` form because it has no error to throw. ### Why are the changes needed? Part of SPARK-56908 (umbrella). The ANSI String->Boolean cast emits an 8-line `if (isTrueString) … else if (isFalseString) … else throw` block in codegen. This PR collapses it to a one-line `UTF8StringUtils.toBooleanExact(...)` call. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? ``` build/sbt "catalyst/testOnly *CastSuite *CastWithAnsiOnSuite *CastWithAnsiOffSuite *AnsiCastSuite *TryCastSuite" ``` 307/307 pass. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor 1.x Closes #55937 from gengliangwang/SPARK-56912-cast-boolean. Authored-by: Gengliang Wang <gengliang@apache.org> Signed-off-by: Gengliang Wang <gengliang@apache.org> (cherry picked from commit 634cda0) Signed-off-by: Gengliang Wang <gengliang@apache.org>

This was referenced May 17, 2026

[SPARK-56909][SQL] Simplify Cast to int/long codegen under ANSI mode #55934

Closed

[SPARK-56910][SQL] Simplify Cast to byte/short codegen under ANSI mode #55935

Closed

gengliangwang force-pushed the SPARK-56912-cast-boolean branch 2 times, most recently from 5bb514b to 5a88ec7 Compare May 17, 2026 23:35

gengliangwang requested review from cloud-fan and viirya May 17, 2026 23:39

gengliangwang marked this pull request as draft May 17, 2026 23:39

gengliangwang force-pushed the SPARK-56912-cast-boolean branch from 5a88ec7 to aee86c1 Compare May 18, 2026 17:07

gengliangwang marked this pull request as ready for review May 22, 2026 22:28

gengliangwang force-pushed the SPARK-56912-cast-boolean branch from aee86c1 to 365cabe Compare May 22, 2026 22:28

gengliangwang mentioned this pull request May 23, 2026

[SPARK-57004][SQL] Simplify CheckOverflowInSum codegen under ANSI mode #56063

Closed

gengliangwang force-pushed the SPARK-56912-cast-boolean branch from 365cabe to b186a2a Compare May 28, 2026 05:57

cloud-fan reviewed May 28, 2026

View reviewed changes

address cloud-fan review: move stringToBooleanExact to UTF8StringUtil…

83a3cbb

…s.toBooleanExact, peer the other string ANSI helpers

gengliangwang requested a review from dongjoon-hyun May 29, 2026 16:08

dongjoon-hyun reviewed May 29, 2026

View reviewed changes

dongjoon-hyun approved these changes May 29, 2026

View reviewed changes

gengliangwang closed this in 634cda0 May 29, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-56912][SQL] Simplify Cast to boolean codegen under ANSI mode#55937

[SPARK-56912][SQL] Simplify Cast to boolean codegen under ANSI mode#55937
gengliangwang wants to merge 2 commits into
apache:masterfrom
gengliangwang:SPARK-56912-cast-boolean

gengliangwang commented May 17, 2026 •

edited

Loading

Uh oh!

gengliangwang commented May 17, 2026 •

edited

Loading

Uh oh!

gengliangwang commented May 21, 2026

Uh oh!

cloud-fan left a comment

Uh oh!

cloud-fan May 28, 2026

Uh oh!

gengliangwang May 28, 2026

Uh oh!

dongjoon-hyun left a comment

Uh oh!

gengliangwang commented May 29, 2026

Uh oh!

dongjoon-hyun left a comment

Uh oh!

gengliangwang commented May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

gengliangwang commented May 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

Uh oh!

gengliangwang commented May 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Stack overview (SPARK-56908 umbrella)

Uh oh!

gengliangwang commented May 21, 2026

Uh oh!

cloud-fan left a comment

Choose a reason for hiding this comment

Uh oh!

cloud-fan May 28, 2026

Choose a reason for hiding this comment

Uh oh!

gengliangwang May 28, 2026

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

gengliangwang commented May 29, 2026

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

gengliangwang commented May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gengliangwang commented May 17, 2026 •

edited

Loading

gengliangwang commented May 17, 2026 •

edited

Loading